Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in partially observable stochastic environment. In standard independent Q-learning approach, performance under partial observability drops at scale environments. Here, combination learning from off-line convex optimisations on historical data isolating marginal contributions total rewards reward signals increases stability scale. Using fixed-size Q-tables, prosumers are able assess their impact system objectives without sharing personal either with each other or central coordinator. Case studies used fitness different combinations exploration sources, definitions, frameworks. It is demonstrated that proposed strategies create value individual levels thanks reductions costs energy imports, losses, distribution network congestion, battery depreciation greenhouse gas emissions.